Deep transfer learning-based gaze tracking for behavioral activity recognition

Computational Ethology studies focused on human beings is usually referred as Human Activity Recognition (HAR). Specifically, this paper belongs to a line of work on the identification of broad cognitive activities that users carry out with computers. The keystone of this kind of systems is the noninvasive detection of the subject's gaze fixations in selected display areas. Noninvasiveness is ensured by using the conventional laptop cameras without additional illumination or tracking devices. The gaze ethograms, composed as sequences of gaze fixations, are the basis to identify the user activities. To determine the gaze fixation display areas with the highest accuracy, this paper explores the use of a transfer learning approach applied to several well-known deep learning network (DLN) architectures whose input is the eye area extracted from the face image,and output is the identification of the gaze fixation area in the computer screen. Two different datasets are created and used in the validation experiments. We report encouraging results that may allow the general use of the system.This work has been supported by FEDER funds through MINECO project TIN2017-85827-P. This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 777720. XinZhe Jin contributed some early computational experiences

An ongoing review of speech emotion recognition

User emotional status recognition is becoming a key feature in advanced Human Computer Interfaces (HCI). A key source of emotional information is the spoken expression, which may be part of the interaction between the human and the machine. Speech emotion recognition (SER) is a very active area of research that involves the application of current machine learning and neural networks tools. This ongoing review covers recent and classical approaches to SER reported in the literature.This work has been carried out with the support of project PID2020-116346GB-I00 funded by the Spanish MICIN

Fusion of probabilistic knowledge-based classification rules and learning automata for automatic recognition of digital images

In this paper, the fusion of probabilistic knowledge-based classification rules and learning automata theory is proposed and as a result we present a set of probabilistic classification rules with self-learning capability. The probabilities of the classification rules change dynamically guided by a supervised reinforcement process aimed at obtaining an optimum classification accuracy. This novel classifier is applied to the automatic recognition of digital images corresponding to visual landmarks for the autonomous navigation of an unmanned aerial vehicle (UAV) developed by the authors. The classification accuracy of the proposed classifier and its comparison with well-established pattern recognition methods is finally reported

Application of self-organizing techniques for the distribution of heterogeneous multi-tasks in multi-robot systems

This paper focuses on the general problem of coordinating of multi-robot systems, more specifically, it addresses the self-election of heterogeneous and specialized tasks by autonomous robots. In this regard, it has proposed experimenting with two different techniques based chiefly on selforganization and emergence biologically inspired, by applying response threshold models as well as ant colony optimization. Under this approach it can speak of multi-tasks selection instead of multi-tasks allocation, that means, as the agents or robots select the tasks instead of being assigned a task by a central controller. The key element in these algorithms is the estimation of the stimuli and the adaptive update of the thresholds. This means that each robot performs this estimate locally depending on the load or the number of pending tasks to be performed. It has evaluated the robustness of the algorithms, perturbing the number of pending loads to simulate the robot’s error in estimating the real number of pending tasks and also the dynamic generation of loads through time. The paper ends with a critical discussion of experimental results

Response threshold models and stochastic learning automata for self-coordination of heterogeneous multi-task distribution in multi-robot systems.

This paper focuses on the general problem of coordinating multiple robots. More specifically, it addresses the self-selection of heterogeneous specialized tasks by autonomous robots. In this paper we focus on a specifically distributed or decentralized approach as we are particularly interested in a decentralized solution where the robots themselves autonomously and in an individual manner, are responsible for selecting a particular task so that all the existing tasks are optimally distributed and executed. In this regard, we have established an experimental scenario to solve the corresponding multi-task distribution problem and we propose a solution using two different approaches by applying Response Threshold Models as well as Learning Automata-based probabilistic algorithms. We have evaluated the robustness of the algorithms, perturbing the number of pending loads to simulate the robot’s error in estimating the real number of pending tasks and also the dynamic generation of loads through time. The paper ends with a critical discussion of experimental results

Improved Activity Recognition Combining Inertial Motion Sensors and Electroencephalogram Signals

Human activity recognition and neural activity analysis are the basis for human computational neureoethology research dealing with the simultaneous analysis of behavioral ethogram descriptions and neural activity measurements. Wireless electroencephalography (EEG) and wireless inertial measurement units (IMU) allow the realization of experimental data recording with improved ecological validity where the subjects can be carrying out natural activities while data recording is minimally invasive. Specifically, we aim to show that EEG and IMU data fusion allows improved human activity recognition in a natural setting. We have defined an experimental protocol composed of natural sitting, standing and walking activities, and we have recruited subjects in two sites: in-house (N = 4) and out-house (N = 12) populations with different demographics. Experimental protocol data capture was carried out with validated commercial systems. Classifier model training and validation were carried out with scikit-learn open source machine learning python package. EEG features consist of the amplitude of the standard EEG frequency bands. Inertial features were the instantaneous position of the body tracked points after a moving average smoothing to remove noise. We carry out three validation processes: a 10-fold cross-validation process per experimental protocol repetition, (b) the inference of the ethograms, and (c) the transfer learning from each experimental protocol repetition to the remaining repetitions. The in-house accuracy results were lower and much more variable than the out-house sessions results. In general, random forest was the best performing classifier model. Best cross-validation results, ethogram accuracy, and transfer learning were achieved from the fusion of EEG and IMUs data. Transfer learning behaved poorly compared to classification on the same protocol repetition, but it has accuracy still greater than 0.75 on average for the out-house data sessions. Transfer leaning accuracy among repetitions of the same subject was above 0.88 on average. Ethogram prediction accuracy was above 0.96 on average. Therefore, we conclude that wireless EEG and IMUs allow for the definition of natural experimental designs with high ecological validity toward human computational neuroethology research. The fusion of both EEG and IMUs signals improves activity and ethogram recognitionThis work has been partially supported by FEDER funds through MINECO Project TIN2017-85827-P. Special thanks to Naiara Vidal from IMH who conducted the recruitment process in the framework of Langileok project funded by the Elkartek program. This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 777720

A vision based aerial rbot solution for the IARC 2014 by the Technical University of Madrid

The IARC competitions aim at making the state of the art in UAV progress. The 2014 challenge deals mainly with GPS/Laser denied navigation, Robot-Robot interaction and Obstacle avoidance in the setting of a ground robot herding problem. We present in this paper a drone which will take part in this competition. The platform and hardware it is composed of and the software we designed are introduced. This software has three main components: the visual information acquisition, the mapping algorithm and the Aritificial Intelligence mission planner. A statement of the safety measures integrated in the drone and of our efforts to ensure field testing in conditions as close as possible to the challenge?s is also included

Modelado de entornos con técnicas basadas en redes de petri borrosas para la exploración y planificación de robots autónomos

El objetivo de esta tesis es el desarrollo de una arquitectura de control de robots autónomos en la que se combinan tareas de navegación, aprendizaje del modelo del entorno mediante exploración y planificación de rutas. La navegación se plantea desde un punto de vista reactivo que dota al sistema con características de robustez y flexibilidad. La planificación de rutas incluye elementos reflexivos sobre los que se apoya la toma de decisiones en la elaboración del plan. La integración de ambos planteamientos supone necesariamente tomar como referencia una aproximación híbrida para nuestra arquitectura de control. Los mecanismos a los que hacemos referencia han sido probados en un robot móvil nomad-200. Los entornos para las pruebas son los laboratorios y pasillos de la facultad de informática de la universidad politécnica de Madrid. The aim of this thesis is the development of a control architecture for autonomous robots that combines tasks for navigating, world modelling through exploration and route planning. The navigation is established from the reactive or behaviour-based point of view that endows to the system with robustness and flexibility. The route planning includes deliberative elements for making decisions in the accomplishment of the plan. The integration of reactive and deliberative elements involves taking the framework of a hybrid approximation for the control architecture. The control architecture is based on a cyclic executive for guaranteeing the real time activities of the robot. The processes constitute the elements of the lowest abstraction level in the architecture. The grouping of processes forms the elements of the second abstraction level: the behaviours. A behaviour is identified as an operation mode of the control system that must achieve a short-term goal. The behaviours can be inhibited, can momentarily stay out of the system. This inhibition mechanism originates the different strategies that can be used for navigating. The strategies are included in the next abstraction level of the control architecture and their actions can be observed at medium-term. The behaviour selection for resolving a same problem with different results defines the elements of the upper abstraction level: the activities. An activity causes an operation mode of the system that can be observed at long-term. We distinguish three kind of activities. In the environment directed activity the robot wanders in the environment without a long-term goal, only the robot must avoid collisions with the obstacles. In the exploration directed activity the robot explores and constructs an environment model. The robot decisions in this activity are determined by the need of improvement the knowledge about the world. In the suggestion directed activity the robot uses the environment model for planning routes to a place. Each activity requires the definition of specific mechanisms. Specifically, we have to resolve the problems of the collision-free navigation, the detection and recognition of reference places in the environment, the definition of schemes that determine the actions during the exploration stage, and the efficient route planning to specific places in the environment. The control architecture and the above mentioned mechanisms have been tested on a NOMAD-200 mobile robot platform. The office-like environments for testing are the laboratories and corridors of the Computer Science Faculty at the Technical University of Madrid